Past Event: Oden Institute Seminar

DeepOSets: Non-Autoregressive In-Context Learning of Supervised Learning Operators

Ulisses Braga-Neto, Professor, Texas A&M

Abstract

We present the DeepSets Operator Network (DeepOSets), a new efficient, non-autoregressive neural network architecture that we introduced recently for in-context learning of operators. In-context learning allows a trained machine learning model to learn from a user prompt without further training. DeepOSets adds in-context learning capabilities to Deep Operator Networks (DeepONets) by combining it with the DeepSets architecture. After briefly reviewing in-context learning and operator learning, we describe in detail the application of DeepOSets to the problem of learning supervised learning algorithms, which are operators mapping a finite-dimensional space of labeled data into an infinite-dimensional hypothesis space of prediction functions. In an empirical comparison with a popular transformer-based model for in-context learning of the least-squares linear regression algorithm, DeepOSets reduced the number of model weights by several orders of magnitude and required a fraction of training and inference time. Furthermore, DeepOSets proved to be less sensitive to noise, outperforming the transformer model in noisy settings. Finally, we intend to describe briefly ongoing multidisciplinary research efforts at the Scientific Machine Learning Lab of the Texas A&M Institute of Data Science (TAMIDS).

 

Biography

Ulisses Braga-Neto received his Ph.D. in Electrical and Computer Engineering from The Johns Hopkins University in 2002. He is currently a Professor in the Electrical and Computer Engineering Department at Texas A&M University. His research focuses on Machine Learning and Statistical Signal Processing. Dr. Braga-Neto is the founding Director of the Scientific Machine Learning Lab at the Texas A&M Institute of Data Science (TAMIDS). He has published two textbooks and more than 160 peer-reviewed journal articles and conference papers. Dr. Braga-Neto received the NSF CAREER Award in 2009.